Data Led Growth project | CARS24

📄

Data Led Growth project | CARS24

Problem Statement:

The primary challenge faced in the Consumer Financing POD of CARS24 is the high drop-off rate during the loan application process. This issue is particularly evident in the conversion rates from lead generation to loan approval, with significant drop-offs observed at multiple stages of the customer journey. Our goal is to analyse the data, identify the key factors contributing to these drop-offs, and design experiments to improve conversion rates.

Overview

From January to April 2023, the loan approval process experienced a noticeable decline in conversion rates. Analysis of 1,500+ bank approvals revealed discrepancies in approval rates and interest rates across different banks. The major contributing factors include:

High deductibles and interest rates for certain customer segments
Varied approval rates across different loan-to-value (LTV) ratios
Significant differences in the approval process efficiency among banks

By diving into the data, we aim to identify actionable insights and propose targeted interventions to improve the overall conversion rates.

Data Insights

Bank	Category	Jan Approval Rate	Apr Approval Rate	Jan Applicant Share	Apr Applicant Share	Rate Impact	Mix Impact	Overall Impact
K-Leasing	Greater than 100% LTV	66%	64%	60%	59%	-2%	-1%	-3%
K-Leasing	100% LTV	18%	17%	18%	17%	-1%	-1%	-2%
K-Leasing	Less than 100% LTV	25%	24%	25%	24%	-1%	-1%	-2%
TTB	Greater than 100% LTV	65%	63%	62%	61%	-2%	-1%	-3%
TTB	100% LTV	8%	7%	8%	7%	-1%	-1%	-2%
TTB	Less than 100% LTV	34%	33%	34%	33%	-1%	-1%	-2%
Krungsri	Greater than 100% LTV	55%	53%	50%	48%	-2%	-2%	-4%
Krungsri	100% LTV	19%	18%	19%	18%	-1%	-1%	-2%
Krungsri	Less than 100% LTV	36%	35%	36%	35%	-1%	-1%	-2%
KKP	Greater than 100% LTV	46%	44%	76%	74%	-2%	-2%	-4%
KKP	100% LTV	3%	2%	3%	2%	-1%	-1%	-2%
KKP	Less than 100% LTV	24%	23%	24%	23%	-1%	-1%	-2%

Key Findings:

K-Leasing: Experienced a minor overall impact of -3% due to slight decreases in both approval rates and applicant shares across all categories.
TTB: Similar overall impact of -3%, with the most significant drop seen in the "Greater than 100% LTV" category.
Krungsri: Notable overall impact of -4%, primarily due to larger decreases in the "Greater than 100% LTV" category.
KKP: Also faced a -4% overall impact, with major drops in both approval rates and applicant shares for "Greater than 100% LTV".

Hypothesis and Experiment

Hypothesis:
To increase the conversion of loan applications, targeted interventions based on customer segmentation (LTV ratios) and bank-specific approval rates will be more effective. Offering tailored loan packages and optimising the customer journey can significantly reduce drop-offs.

Null Hypothesis:
Offering bundled discounts on loans and additional benefits will not increase their conversion rates.

Goals:

Increase the loan approval rate by optimising the customer journey and loan packages.
Reduce the drop-off rate at each stage of the loan application process.

Experiment Design:

What Are We Testing?
We are testing the effectiveness of three different interventions on loan approval rates based on customer segmentation.

Variation Design:

Group A (Reduced Deductible for High LTV Customers):
- Hypothesis: Reducing the deductible for customers with greater than 100% LTV will increase approval rates.
- Why: High LTV customers face higher upfront costs, which can deter approval. Reducing deductibles from the average of 4.2% to 2.1% can make loans more attractive and affordable.
Group B (Special Low-Interest Rate for 100% LTV Customers):
- Hypothesis: Offering special low-interest rates for 100% LTV customers will increase approval rates.
- Why: 100% LTV customers have moderate risk and providing a lower interest rate (e.g., from 6.3% to 3.5%) can incentivise more customers to apply, improving approval rates.
Group C (Additional Incentives for Less than 100% LTV Customers):
- Hypothesis: Providing additional incentives such as longer loan tenures or lower down payments for customers with less than 100% LTV will increase approval rates.
- Why: Less than 100% LTV customers are the least risky. Offering longer loan tenures (e.g., from 72 months to 84 months) or lower down payments can enhance affordability and encourage approvals.

Control Group: No additional offers or changes in the current loan packages.

Supporting Evidence:

Historical data showing drop-off rates and approval rates.
External benchmarks indicating the impact of similar interventions.

Sample Size Calculation:

To detect a significant effect in conversion rates, we will use the following sample sizes for the experiment:

Group A (Target Increase 10%): 2,562
Group B (Target Increase 8%): 3,875
Group C (Target Increase 5%): 6,414

Calculation Parameters:

Confidence Level: 95%
Power: 80%
Effect Size: Varies based on the target increase in approval rates (5%, 8%, 10%)

Duration of the Test:

Apr 24th to May 12th or until statistical significance is achieved.

Success Metrics

Metrics:

Increase the loan approval rate by 10% for Group A.
Increase the loan approval rate by 8% for Group B.
Increase the loan approval rate by 5% for Group C.
Reduce the drop-off rate at each stage of the loan application process.

Defining Success Metrics:

Worst Case: Absolute increase in conversion by 1.5%
Best Case: Absolute increase in conversion by 5%

A/A Test:

Purpose: To ensure the experimental setup does not introduce any biases or unexpected variations.
Design: Split the existing customer base into two groups (A1 and A2) without any changes and compare their behaviour.

Experiment Results

Actual Data and Calculations:

Group	Dec'23 Conv%	Mar'24 Conv%	Apr'24 Conv%
Control	16.20%	12.00%	12.00%
Group A	16.20%	12.00%	16.00%
Group B	16.20%	12.00%	15.50%
Group C	16.20%	12.00%	14.00%

Experiment Result:

The experiments demonstrated that targeted loan packages and customer segmentation significantly improve loan approval rates. Group A, with reduced deductibles for high LTV customers, showed the highest increase in conversion rates.

Release decision:

Scale: The experiment for Group A was successful with statistical significance. We will scale this intervention across all customer segments that fall under the high LTV category.

Kill: The experiment for Group C was unsuccessful with statistical significance, showing minimal improvement. We will discontinue this intervention.

Continue: The experiment for Group B showed positive results but did not reach statistical significance. We will continue running this experiment for further validation and potential adjustments.

Core Learnings:

Group A: Reduced deductibles for high LTV customers effectively increased approval rates by 5%.
Group B: Special interest rates for 100% LTV customers led to an increase of 3%.
Group C: Additional incentives for less than 100% LTV customers resulted in a 1% increase in approval rates.

Next Steps

Action Items:

Scale the Successful Experiment (Group A): Implement the reduced deductible intervention for all high LTV customers. Document technical specs, designs, timelines, and ramp-up milestones.
Continue Group B Experiment: Refine the special low-interest rate offer and extend the experiment duration to achieve statistical significance.
New Experiments for Group C: Develop new hypotheses based on the learnings. Possible next experiments could include targeted marketing for less than 100% LTV customers or different types of incentives.

Stakeholder Management Plan

Stakeholder Level	Stakeholder Role	When to Communicate	What to Communicate	Method of Communication
Level 1	CEO, VP Business	Initial Planning, Mid-Experiment, End of Experiment	Project objectives, alignment with business goals, interim results, final results, next steps	Meetings, Executive Summaries
Level 1	Product Head	Initial Planning, Weekly Updates, End of Experiment	Detailed experiment plan, weekly progress reports, final results, and learnings	Meetings, Detailed Reports
Level 2	Data Team	Initial Planning, Weekly Sync-ups, Post-Analysis	Data collection requirements, analysis methods, interim data findings, final data insights	Meetings, Collaboration Tools
Level 2	Marketing Team Leads	Initial Planning, Monthly Updates, End of Experiment	Overview of experiment, potential marketing impacts, final results, and recommendations	Meetings, Email Updates

How Buy-in Achieved for This Project

Step	Actions	Details
Initial Engagement	Problem Identification	Presented data highlighting significant drop-offs in loan application conversions.
	Solution Proposal	Outlined hypotheses and potential interventions.
Clear Communication	Detailed Documentation	Provided comprehensive documentation on experiment design, success metrics, and expected outcomes.
	Visual Aids	Used charts and graphs to illustrate current state and expected improvements.
Regular Updates	Progress Reports	Regular updates on experiment progress, interim results, and adjustments.
	Transparency	Shared both positive and negative results, discussing learnings and involving stakeholders.
Demonstrating Impact	Alignment with Business Goals	Emphasized alignment with broader business objectives (increasing approval rates, reducing drop-offs).
	Potential Benefits	Highlighted potential benefits such as improved customer satisfaction and increased revenue.
Collaborative Approach	Stakeholder Meetings	Scheduled regular meetings to discuss the project, gather feedback, and ensure alignment.
	Inclusive Decision-Making	Involved stakeholders in critical decisions to ensure buy-in and support.

Conclusion

The primary challenge faced in the Consumer Financing POD was the significant drop-off in loan application conversion rates. Through targeted experimentation, we tested various interventions to address this issue.

By reducing deductibles for high LTV customers (Group A), offering special low-interest rates for 100% LTV customers (Group B), and providing additional incentives for less than 100% LTV customers (Group C), we identified the most effective strategies to increase approval rates.

Group A showed a significant improvement, with approval rates increasing from 12.00% to 17.00%, which will be scaled across similar segments. Group B saw a positive but not statistically significant increase, and will continue to be refined. Group C had minimal impact and will be discontinued.

This data-driven approach successfully addressed the problem, providing actionable insights and demonstrating the value of targeted interventions in improving loan approval rates.

Brand focused courses

Great brands aren't built on clicks. They're built on trust. Craft narratives that resonate, campaigns that stand out, and brands that last.

View all courses